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We study a mixture of ultracold spin-half fermionic and spin-one bosonic atoms in a shallow 
optical lattice where the bosons are coupled to the fermions via both density-density and spin-spin 
interactions. We consider the parameter regime where the bosons are in a superfluid ground state, 
integrate them out, and obtain an effective action for the fermions. We carry out a renormalization 
group analysis of this effective fermionic action at low temperatures, show that the presence of the 
spinor bosons may lead to a separation of Fermi surfaces of the spin-up and spin-down fermions, and 
investigate the parameter range where this phenomenon occurs. We also calculate the susceptibilities 
corresponding to the possible superfluid instabilities of the fermions and obtain their possible broken- 
symmetry ground states at low temperatures and weak interactions. 

PACS numbers: 



I. INTRODUCTION 

The remarkable experimental achievements in the 
field of ultracold atom physics have made it possible 
to generate mixtures of fermionic atoms with different 
spin populations^, as well as mixtures of fermionic and 
bosonic atoms in a trap^ that can also be loaded on op- 
tical lattices-. Multi-species fermions with unequal den- 
sities have also been extensively studied not only in cold 
atom systems, but also in electronic materials, such as 
the magnetic-field induced organic superconductors^ and 
other correlated fermion systems^, as well as in the con- 
text of color superconductivity in dense quark matter—. 
Bose-Fermi mixtures present a rich phase diagram and 
have also been subject of intense researcbi"— . Several 
studies on such mixtures have been restricted to either 
one-dimensional systems* or to cases where the coupling 
between the bosons and the fermions are weaki^"— . The 
existence of a supersolid phase in these system in such 
a weak coupling regime has been predictectii. Phase 
separation^ and phases with staggered currentsi^ have 
also been investigated. Some of the other studies^, 
which have looked at the strong coupling regime, have 
restricted themselves to integer filling factors of bosons 
and fermions or considered a description of these sys- 
tems at half filling either by using analytical slave-boson 
mean-field technique^^ or numerical dynamical mean- 
field theory ^^. 

An interesting aspect of the study of quantum mix- 
tures is that one species of atoms may mediate inter- 
actions among atoms of the other species. In a Bose- 
Fermi mixture where the bosons form a Bose-Einstein 
condensate (BEC), quantum fluctuation of the BEC can 
mediate long-range attractive interaction between the 
fermionsi^^— . Conversely, in another regime, fermions 
can be viewed as mediating an effective long-range in- 
teraction between the bosonic atoms^. In this work we 
investigate the problem of partially polarized fermions 
(unequal spin populations) in the presence of mediated 



interaction due to quantum fluctuations of a BEC of 
bosonic atoms. Starting from fermions with equal spin 
populations, we show that the spin asymmetry of the 
fermion filling factors can arise due to coupling to a 
spinor BEC. Spinor boson BEC systems have been stud- 
ied both experimentally^* and theoreticall y^^i^° . In par- 
ticular, the phases and low-energy excitations of such a 
system is well-known. Here we consider the effect of cou- 
pling of these excitations to the fermionic atoms in the 
mixture. 

An important tool for understanding the phases of 
interacting fermions is the renormalization group (RG) 
technique^ ^. It has been applied to study the phase dia- 
gram of a Bose-Fermi mixture with fermion interactions 
mediated by fluctuations of the boson BEC, on square 
and triangular lattices'^ ^. The RG for fermions has also 
been extended to frequency-dependent interactions^^, 
where retardation effects are import an t^^i^^ . In this 
work, we use the renormalization group technique to 
study a mixture of ultracold spin- half fermionic and spin- 
one bosonic atoms in a shallow optical lattice in two di- 
mensions where the bosons are coupled to the fermions 
via both density-density and spin-spin interactions. The 
main aim of our study is to understand the effect of an 
inter-species on-site SU(2) invariant spin-spin interaction 
on the phases of this system. We consider the parame- 
ter regime where the interaction between the bosons and 
the fermions are weak and the bosons are in a superfluid 
state. Wc then start with a mean-field treatment of the 
bosons, and include quantum fluctuations to flrst order 
within a 1/A'^ approximation. After a suitable Bogoli- 
ubov transformation, the bosonic modes are integrated 
out, and an effective action for the fermions is obtained. 
We find that, when the bosons are in a spinor super- 
fiuid state, the spin-spin interaction leads to an effective 
fermionic action with shifted Fermi surfaces for the up- 
and down-spin fermions. We then carry out a renormal- 
ization group analysis of this effective fermionic action 
at low temperature and chart out the fate of such a shift 



under RG flow for different parameter regimes. We also 
calculate the susceptibilities corresponding to the super- 
fluid instabilities of the fermions and obtain the possible 
broken-symmetry fermionic ground states at low temper- 
ature and weak interactions. In particular we show that 
the leading instability for the fermions with attractive in- 
teraction and circular Fermi surface occurs in the triplet 
superfluid channel. 

The organization of the rest of the paper is as follows. 
In Sec. ini we introduce the model Hamiltonian for the 
Bose- Fermi mixture and derive the effective fermionic ac- 
tion. In Sec. mil we obtain the RG equations for the 
fermionic self energy and interactions from this action. 
Next, in Sec. lIVi we analyze the RG flow of different sus- 
ceptibilities. Finally, we present a discussion of our main 
results and conclude in Sec. El 




FIG. 1: Schematic representation of the separation between 
the up and down spin Fermi surface. SKp = [Kp:^ — Kpf) 
is the difference between the magnitudes of the up and the 
down spin Fermi momenta. 



II. EFFECTIVE FERMIONIC HAMILTONIAN 

The Hamiltonian of a Bose-Fermi mixture in a shallow 
ultracold lattice is given hy H — Hp + Hb + Hbf- The 
fermionic part of the Hamiltonian Hp is given by 



Hf = Y^{e^^- t^F)T^„ha 



ka 



/ , '^q.'k'-q,cr/k',CTJl(.+q,CT-/k,<T 
q.k.k'.cr 

/ . '^q/k'-q,cr/k',CT/k+q,CT/k,ffj 
q,k,k',(T 



(1) 



where /ko-C/kg.) is the annihilation(creation) operator 
for the fermions, fip denotes their bare chemical po- 
tential (taken to be independent of the spin of the 

fermions), C/q denotes the bare interaction between 
the fermions on the same (separate) Fermi surfaces, 
eic — —2tp[cos{kxa) + cos{kya)] is the fermion dispersion, 
tp is the hopping amplitude of the fermions between the 
neighboring sites, a =1 (f) for a =t (i), and a is the lat- 
tice spacing. For later use, we define the two-component 
fermionic field ((>{ = {fif, fiif^ and use it to represent the 



and number den- 
denotes the Pauli 



fermionic spin-density S^ ~ (f>l(T^(j), 

sity nf = (j)\(j)i, where a = {(7x,c'y,(^z, 
matrices. 

The Hamiltonian Hb for the spinor bosons is given 
by20 



Hb = -h E ^l^. 



■JQ ' —^ 7 ,"ta("-M ~ 1) 



<i,j>,a 



Ub2 



^((Sf)2-2nfj-^^^nf„, 



(2) 



where a — —1,0, 1 denotes the azimuthal spin quantum 
number of the bosons, 6^ {nf^ = h\j)ia) is the bosonic 
annihilation (density) operator, tb is the boson hopping 
amplitude between neighboring sites, Uho and Uh2 denote 



the on-site boson interaction strengths in the spin-0 and 
spin- 2 channels respectively, and jib is the chemical po- 
tential for the bosons. The spin density of these bosons 
can be expressed in terms of the generators of spin-one 
matrices: Sf — bl^Xapbip. The detailed expression for 
the generators A is given in Appendix |^ 

The most general SU(2) invariant on-site interaction 
between the bosons and the fermions is represented by 
Hbf- Note that since the fermions carry spin half, con- 
servation of azimuthal quantum number m^ does not pre- 
clude an on-site spin-spin interaction between the bosons 
and the fermions. Thus we consider the Hamiltonian 
Hbf to be of the form 



H 



BF 



Ussj:si 



Udd ^ nfnf 



(3) 



In what follows, we are going to consider the parameter 
regime Uss <C Udd 7^ 0. We note that the presence of a 
non-zero Uss is a key feature of the subsequent analysis 
carried out in this work. 

The analysis of the coupled Bose-Fermi system is most 
easily done in terms of coherent state path integrals. Fol- 
lowing standard prescription, we write the partition func- 
tion of the system as 



S = Sb + Sp + Sbf, 






J2 [fc*(k, nn)in^b{k, n,,)] - Hb [b*,b 

k 

J2[nK^n)lOJnf{k,U;„)]^Hp[rj] 



Sbf — ^ 7 ^ Hbf, 



(4) 



where b = {bi,bo,b^i) [f = (/-f,/^)] denotes bosonic 
[fermionic] fields, r2„ i^n) denote bosonic (fermionic) 



Matsubara frequencies, and /3 = l/fc^T with T being 
the temperature, and ks the Boltzniann constant. 

We begin with the analysis of Sb- We transform the 
Hamiltonian written in Eq. ([2]) into x,y, z basis by using 
the following relations 



1 



T2^'-^-'*^^ 



V2 



(b-i+K) 



(5) 



We assume that the bosonic spinor system is deep in the 
BEC state. The standard procedure for analyzing such 
a BEC involves expressing the bosonic field b as 



ba{q,i^n) = ^OaSci,0 + aa{q,iQn), 



(6) 



where ^oa — {bail — 0, fin — 0)), and expanding Sb to 
order 0{a^). The mean- field equation for the condensate 
is then obtained by imposing the coefficient of uq and aj 
to be zero. This analysis yields 



* 



Oq 



-44 



no 



Ubo + (no - l)Ub2 - ^J.b 
-^E(*S/^)'*O" = 0' (7) 



/3#a 



where we have introduced the condensate density no — 
Nb/N, with Nb being the total number of bosons in the 
system, and TV the total number of lattice sites. As shown 
in a previous study, Eq. ([T]) supports two solutions^. The 
first is a ferromagnetic phase with 



^ 



fcrro 



^ ^noN/2{l,±i,0)' 



(8) 



while the other is a polar phase with \l/poiar — 
\/nQN {1,0,0)'^. The ferromagnetic state becomes en- 
ergetically favorable for Ub2 < and in the rest of the 
paper we concentrate on this regime and work with the 
solution *fcrro = \/no^/2(l, «, 0)"^ which corresponds to 
(61) ^ 0. The choice of one of the these two solutions can 
be easily seen to be the effect of any stray magnetic field 
that might be present in a realistic experimental system. 
We note that for the polar phase, the low-energy physics 
of the Bose-Fermi mixture is identical to its counterpart 
with spinless bosonai^ii^. 

A detailed analysis of Sb when the bosons are in 
the ferromagnetic phase is carried out in Appendix |X] 
and leads to the expression for Sb [Eq. (lAip ] which is 
quadratic in the fluctuation fields aa(q, jri„). Thus, us- 
ing Eq. (JA1[) and Eq. (jH]), one can integrate out the bo- 
son degrees of freedom and obtain, after a straightfor- 
ward but tedious calculation, an effective action for the 



fermions, 



P 



itOn k 

/ ^ ^q,crcr/k+q,o-/k,cr/k'-q,fT/k',CT 
k:,k',q,cr 

k,k',q 



where ^jLpa = Mf - ['^oC/ss(Sgncr) -I- Uddn-o], and Sgncr = 
1(— 1) for a =t (i)- Thus the spin-spin interaction be- 
tween the bosons and the fermions leads to an effective 
shift between the spin-up and spin-down Fermi surfaces, 
as shown in Fig. [1] , at the mean-field level. The sign of 
the shift depends on the choice of one of the two solutions 
given by Eq.[5] for our choice ^fcrro = \/noA^/2(l, «, 0)-^, 
the down-spin Fermi surface is enhanced compared to the 
up-spin one as shown in Fig.[TJ The effective interactions 
q,t4, are given by 



C^q,aa and U, 



JJ — U 



[/< 



q,n 



u. 



m. 

2 

no , 



{Udd + {SgIia)Uss)\q,aa, (10) 

' ' 7(U!d~Ul)xci,i:U (11) 



where 



X<},<t<t' 



u 



q,crcr' 



2Ub2no 



iUaa' - 2Ub2no)^ + nl ' 



and ^q,crcr' = -2tb[cos{q, 
is the boson dispersion 

qx[v]aa = KFaCos{9)[ain{9)] 
K Fa) cos{e)[sm{0)]/ 2 



xcrcr' ) 

at 



(12) 
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COs{qyaa' 

the wave-vector 

qx[y]<7<7 = {Kpa + 

Here we have set the lattice 



spacing a = 1, Kp^ is the magnitude of the Fermi wave- 
vector for electrons with spin ct, and we have restricted 
ourselves to the regime where | Kp^ — Kpi \ <^ Kp^ , Kp^ . 
This restricts the validity of our analysis to the param- 
eter regime fip ^ Uss- Note that Uq,fi represents the 
amplitude of scattering between fermions on separate 
Fermi surfaces, while scattering processes represented 
by U. 



q,crcr 



involve fermions on the same Fermi surface. 
For the rest of the paper, we shall ignore the retardation 
effects of the effective interaction and shall thus set 
fin = and restrict ourselves to circular Fermi surfaces 
with small effective shifts between them (as shown in 
Fig. p. 

Next, following standard procedure outlined in Ref. 
\2u . we antisymmetrize Uq^aa with respect to the inter- 
change ki o k2 and ka o k4 (where, ki = k', k2 = 
k^^ ka — k' — q, k4 = k-l-q). Further, following Ref. 
I21I and using the circular nature of the Fermi surface, we 
consider fermion scattering only in the forward (ki = k4 
and k2 = ka) and BCS (k2 = — ki and k4 = — ka) chan- 
nels. The contribution of C/q.o-o- to these channels can be 
computed from Eqs. pT|) and p^ . Denoting interaction 
couplings in these channels by Fccr{6i2) and VaaiOis), 



respectively, we find that 



Faai^u) — 



V^AOu) 



Uq -no 



Um + (Sgn(T)[/, 



2noUb2 



K 



Fa 



x[l - COs(6'i2)] 



[/o-rT-o 



/t/dd + (Sgncr)[/^ 



V 2noUb2 



Ki 



X COs(6'i3), 



(13) 



(14) 



where, ^12(^13) is the angle between ki and k2(k3), 
[/o(l — cos(6'i2))[L/o cos(6'i3)] denote the value of U^ for 
the forward[BCS] channels. Note that in obtaining Eq. 
([T^ and (HU, we have explicitly antisymmetrized the 
contribution of Xq.o-o- in Eq. ([TT 



The contributions of t/q^-^j. in the forward and the BCS 
channels can also be computed in a similar manner and 
are given by 



%(^13) 



Mujd 



ul) 



4K2.,(l-cos(0i2))-4noC/( 



'F,i:i\ 



62 



Muja 



Ul) 



4if2.^^cos(6li3)-4noC/b2' 



(15) 



where, if_F,t4. = (-^Ft + Kpi)/2 and we have set the 
contribution of C/q to the forward and BCS channels to 
zero. We have checked explicitly that finite value of U^ 
does not alter the qualitative conclusions of the work. 



III. RG EQUATIONS FOR SELF-ENERGY AND 
COUPLINGS 



In this section, we carry out a RG analysis of 5off 
adapting one-loop Wilsonian RG using a path integral 
approach. The details of this approach are outlined in 
several past works^tiS. The key idea behind such a pro- 
cedure is to consider ScS as the starting fermionic ac- 
tion at a high-energy cutoff scale A, perform Wilson RG 
on this action and derive the flow equations for the ef- 
fective interactions and fermionic self-energy. It is well 
known^i that for a circular Fermi surface as considered 
here, only the interaction in the BCS channels {V) flow 
under RG and that the key contribution to the fermionic 
self-energy within one-loop RG comes from the forward 
channels {F). 

Using these facts, the relevant diagrams for the contri- 
bution to the self-energy and the effective interactions in 
the present model [Eq. ^] can be easily found. These 
are shown in Fig. [21 The RG equations for the inter- 
actions and the fermionic self-energy, as obtained from 
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FIG. 2: (a) Schematic representation of the effective fermionic 
interaction U (left) as sum of the bare interaction f/o and 
the contribution from the bosons, (b) Diagrammatic repre- 
sentation of the RG flow equation for the self-energy (Ec) 
correction which receives contribution from the interaction in 
the forward channel F. (c) Diagrammatic representation for 
the RG flow equations for V . We use the simplified notation 
1 = k', 2=k, 3 = k' — q, 4 = k -|- q, and a and a' takes values 
t and 4.. 



these diagram in Fig. [2l are given by 

(Ea{e) 



-TT / (KpaF^^d' - 9)GaiLUn,d') 

+KFaF^^{e' -e)Gs{^n,e')), (i6) 



dVaa{ei-ez) 

di 

dVniei - 03) 



Kpa 

27r 



v„,{ei - e)v,,{e ~ 6:,) 



xGa{uJn,0)Gai-UJn,0 + TT), 



(17) 



K 



FU 



27r 



euj„ 



VniOi ~ e)Vnio ~ 63) 



xG;(w„,6i)Gt(-w„,6i + 7r), 



(18) 



where we have carried out the integrals over the radial 
momentum perpendicular to the circular Fermi surface, 
A — Aqc^^ is the RG cutoff, Aq < Ep is the cut- 
off in the beginning of the RG flow, £ is the RG time, 
le'ui ~^/f^^iuj /rf^V (2"') denotes frequency sum and 
integral over transverse momenta over the Fermi sur- 
face, Scr(0) denotes the self-energy for fermions with 
spin a and momentum k = {KFcrCOs{9),KFaSm{9)), 
KF,aa = {Kpi; + Kp^j^, ta{9') is the fcrmiou disper- 
sion on the Fermi surface with spin a , and the fermion 
Green function, evaluated on the Fermi surface for spin 
a electrons are given by 



G,(w„,0) = (^c^„ - (e,(0) - ^Xa) - 2,(0))" 



(19) 



Before solving Eqs. ((16)) . . ([T8 | numerically, we note that 
^a{0) — —2t[cos{KFaCO&{S)) -\- cos{K p a ^^^{9))] have 
a very weak 9 dependence. Further, the integra- 
tion Fa-a'{9' — 9) over 9' for a complete cycle renders 
it independent of 9 as well. Consequently, So- be- 
comes independent of 9. Thus, at low temperature, 
^^ Gcr(a;„,6')Gcr'(— w„,0 -I- tt) becomes practically in- 
dependent of 9. Using this fact, it is possible to express 



Eqs. p6|) .. (|T8)) in the angular momentum channels de- 
noted by I to obtain 



-^ = -—(Kp^Fi^G', + Kp-^F\^G',), (20) 



d£ 



J^Fa-Jaa /f^/ \2 



27r 



-iv^.r 






di 277 

where Vj^, [F^^^, ] and G^„ are given by 



(21) 
(22) 

(23) 



iuJn 

-^n - i5]Gt(c.„,0)G^(-w„,0 + ^). (24) 



Next, we solve the RG equations for self energy and cou- 
plings numerically for a temperature /Stp — 10. We 
have carried out the numerical solution of both Eqs. 
([16l)..([l8]) and Eqs. ((20]) . . dm and checked that these 
yield identical results confirming our observation on the 
absence of 9 dependence of Jaa and J^^. For the nu- 
merical solution of these equations, we have scaled all 
the energy parameters in units of 2tF- We note that 
at the one-loop level, the effect of So- is to renormalize 
the chemical potential fipa and hence their difference: 
Si2 = l/iFt - ^J■Fl + S^ - Sj^l = \S^j.i„t -I- S| - Ej^l, where 
SfJ^mt = ^-l■F^ - Mfj, = -2noUss- In Figs. [3] and [H we 
show the variation of Sfi as a function of the RG time £ 
for Uq — ±0.3, no = 0.9, Ub2 — —0.08 and three different 
representative values of (S/imt . We find that the RG fiow 
is essentially controlled by the induced interaction part of 
Fa-a-' and displays little dependence on C/q- The separa- 
tion between the Fermi surfaces is always amplified and 
the spin-up and spin-down Fermi surfaces flow away from 
each other. This signifies a possibility of either a Ferro- 
magnetic or triplet superfluid (with equal-spin pairing) 
instabilities of the system. Note that such instabilities, 
in case they occur, have their root in the initial separa- 
tion of the opposite spin Fermi surfaces and hence can be 
attributed to the spin-spin coupling between the fermions 
and the spinor bosons. 

Next we plot the variation of the couplings with the RG 
cutoff £ in Fig. [5] and Fig. |6] for Sfiint = —0.05, no = 0.9, 
Ub2 = -0.08 and Uq = -0.3. We find that % does not 
flow appreciably under RG which is a consequence of lack 
of scattering between Fermi surfaces with opposite spins. 
The flow of Va-a- shows an increase of their magnitude 
indicating a flow toward strong coupling regime which 
can not be accessed by our perturbative RG analysis. 
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FIG. 3: Plot of 5/1 versus £, for / = 0, no = 0.9, Ub2 = -0.08, 
Uo = —0.3 for several values of Sfiint = —0.05 (green line with 
triangles), 5jj,int ~ —0.03 (red line with circles) and S/iint = 
—0.01 (black line with squares). 
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FIG. 4: Same as in Fig. |3] but with Uo =0.3. 



IV. RG FLOW OF THE SUSCEPTIBILITIES 

In this section, we consider the RG flow for the possi- 
ble instabilities of the fermionic models. In particular, we 
consider the singlet and equal-spin paired triplet super- 
fluid (SSF and TSF) instabilities^i of the metallic phase 
of the fermions due to the induced interaction. This 
choice is motivated by the fact that for circular Fermi sur- 
faces considered here we do not have nesting and hence do 
not expect to have instabilities in the 2kp spin- or charge- 
density wave channels. It is well known that the onset of 
such instabilities are signalled by the divergence of the 
corresponding static susceptibilities under RG flow^. 
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FIG. 5: RG flow of the couplings V^^ (black line with squares), 
Vii (red line with circles) for / = ±1, [/o= -0.3, Ub2 = -0.08, 
5tJ-int ~ —0.05, and no — 0.9. 
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FIG. 6: RG flow of the coupling V^i (black line with squares 
for 1=0), V-[i (red line with circles for I = ±1) for Uo= -0.3, 
Ub2 = -0.08, Sum = -0.05, and no = 0.9. 



The flow equations for the static susceptibilities can 
be derived using standard techniques as elaborated in 
Refs. Il6ll23.25. As outlined in these works, the response 
function can be calculated by introducing a source term 
in the action 



Su = -Y.h'^'. 



(25) 



where 5 takes values SSF or TSF corresponding to the 



singlet or equal-spin triplet pairings. 



.SSF 



A 



TSF 



^(sgncr)f^(k)f<j(-k) and 

cr,k 
/,(k)/,(-k), 



(26) 



are the order parameters for singlet and triplet superflu- 
idity respectively and h is the external field of type 5. 
The corresponding response function is given by 



X 



(A'^*A^) 



5(2)lnZ[h] 



5hS*5h^ 



h=0 



(27) 



where Z[h] denotes the partition function in presence of 
the source term. The RG process generates correction to 
the source field h along with the higher order terms in 
the source field. At any RG time i, the total action Si 
can be written as 



Si = Sf 



dt z^h^A^ 



h' hiq)x' 



(28) 



where S^ is the action at RG time £ without the external 
field h and the coefficient z^ is the effective vertex of type 
S. 

The relevant one-loop diagrams representing the RG 
equations for the vertices z^ and the susceptibilities x^ 
are schematically shown in Fig. [T] The corresponding 
one-loop flow equations for z^ and x* are given by 
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SSFn2 Y^ Kp.aa 



^^ 27r 



'J (7(7 1 



(29) 



where we have used the 9 independence of J^a and J^^. 
We solve these equations numerically for fitp = 10. The 
results are shown in Fig. [8] for C/q — —0.3 for SSF and 
TSF instabilities. We find that the x[f± instability shows 
a divergence around £ ~ 80 indicating an instability of 
the metallic ground states against triplet down-spin pair- 
ing superfluid ground state. This is an expected conse- 
quence of the growing separation of the Fermi surfaces 
which prevents opposite spin SSF pairing and hence fa- 
vors down-spin TSF state. Thus we conclude that the 
most dominant instability of the Fermi superfluid with 
an attractive interaction is TSF with down-spin pairing. 
We note that our RG analysis can not predict the sub- 
sequent fate of the system once the superfluid instability 
has set in. The system may either end up with a spin-up 
metallic Fermi surface coexisting with a triplet superfluid 
of spin-down fermions or the superfluidity in the spin- 
down channel may induce a superfluid instability for the 



K,(T 



-K,cr' -K,a' ^ 
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K,cr 



dX 
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FIG. 7; Diagrammatic representation of the renormalization 
of vertices and the susceptibiUties in the BCS channel. 




FIG. 8: RG flow for the static susceptibilities for f/o = —0.3. 
The blue line with triangle and the black line with square 
indicate RG flow of the susceptibilities for the triplet BCS 
channels yj'^ (I = ±1) and X?^tt (^ ~ -'-^) respectively. The 
red line with circle indicates RG flow of the susceptibility for 
the singlet BCS channel xf^^ (^ = 0). 



spin-up fermions via a momentuni-space proximity ef- 
fect. The latter effect is somewhat similar to that seen 
for multi-band ruthenate superconductors^^. We leave a 
more thorough analysis of these possibilities as a subject 
of future study. 



V. CONCLUSION 

In conclusion, we have studied a mixture of spinor bo- 
son and fermion in a shallow 2D optical lattice using RG 
and have shown that the presence of an on-site spin-spin 
interaction between the bosons and the fermions leads 
to a separation of Fermi surface of the spin-up and spin- 
down fermions irrespective of the nature of the bare in- 



teraction between the fermions provided that the bosons 
are in the spinor condensate state. Such a separation, de- 
pending on the density of the fermions, may give rise to 
a net spin polarization for the Fermi superfluid. Further, 
for attractive interaction between these fermions, we have 
shown that the leading instability of the metallic state of 
the fermions lies in the TSF channel with down spin pair- 
ing. In particular, we predict that for fermions coupled 
to a spinor bosonic condensate in its ferromagnetic phase 
via a spin-spin interactions, attractive interactions will 
induce a down-spin triplet pairing superfluid instability 
over the otherwise more common singlet pairing instabil- 
ity. We note that this phenomenon is in contrast to the 
fermions coupled to either a spinless boson condensate or 
a spinor boson condensate in its polar phase. 
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Appendix A: Effective Quadratic Hamiltonian for 
spinor boson 

The generators A+, A_ for spin-one bosons can be ob- 
tained from the spin-rotation matrices m a;,y and z basis. 
In this basis we have 

/010\ /O-iO 

^VO 1 O; ^lo z 

/ 1 
and 5^ ^ 
\0 -1 
This yields, using A± = ^^ ± iSy 

/O n/2 \ / 

A+ = ^/2 , A_ - n/2 
\00 0/ \0%/20 

and X-, = Sz- 

The action Sb when the bosons are in the ferromag- 
netic phase can be written using Ref. [2] and 2] as 

Sb = --^ |^^[a*(k, fi„)zrj„aQ(k, fi„) 



^H'B[a*,a] 



(Al) 



Hg is the quadratic Hamiltonian for spinor boson in fer- 
romagnetic phase and given by. 



k.a 



no 
2 



no , 



H'b = -^{Um + Ub2)N + ^[a + ^(1 - <5a.)(t/6o - C/fa2)]aLato " ^(^w + SC/^a) ^ iia^aky ~ 4j,«fc^) 



no 



(f/bo + Ub2) 2_^i°-kxa-kx + aj,^a_^^ - auya^ky - a\ya_f.y) + — (t/ho + C/62) ^J ^("L'^-fca ~ o-kxa-ky), (A2) 



where , ^j, = ej, + 4ifc = —2tb{cos{kxa) + cos{kya) — 2). 
We then decouple the boson fields using Eq. ([7]) and ex- 
pand about the ferromagnetic condensate saddle point to 
obtain the quadratic effective action for the bosons. This 
action has the form 

^B = -^^^^*(k,a„)G5U(k,f7„), (A3) 



P 



UJn k 



where .4(k, f2„) denotes the fluctuating boson fields given 
by 



A*{k,rt„) = [<(k,rj„),a,(-k,f]„),a;(k,ao, 

aj;(-k,ri„),a*(k, ri„),a^("k, fi„)] 



(A4) 



and Gg^ denotes the boson Green's function given by 
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ek+y 



where, Pk± = Ck + ^ (Ubo ~ ^62) ± «^n , Ck± = ?k ± i^n, 



A 



^(C/feo 



C/62) and P = ai (C/^o + 3[/^2). Using Eq. 



(IAf[) and Eq. ([3]), we integrate out the bosons by fol 
lowing standard technique and obtain effective fermionic 
action as written in Eq. © . 
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